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A correlation follower comprising an image 
sensor (1) adjustable sideways and in height with a 
limited field of view, which is cyclically scanned 
by the sensor the output signal of which reflects 
the image content within the field of view and a 
video correlator (3) for controlling the alignment 
of the sensor in dependance of the output signal of 
the image sensor. The video correlator (3) has two 
addressable memories (12, 14) the one of which is 
a real time memory (12), i.e. in this memory a 
section of the field of view is stored for each cycle. 
The other memory is a reference image memory 
(14) and is updated with the content of the real 
time memory (12). During the correlation the 
contents of the memories are displaced in relation 
to each other and an error signal, corresponding to 
the position of displacement for which maximum 
correlation is achieved, is made to control, through 
said control circuits, the alignment of the sensor. 
To make the tracking process insensitive to dis- 
turbandes and image elements which appear 
momentarily in the field of view, for each scarming 
cycle only a part of the positions of the refer- 
ence image memory (14) are updated by selecting 
the addresses to said positions randomly or ac- 
cording to a piedeteimined rule of selection. By 
that the positions which are updated during one 
cycle in positions are distributed over the entire 
area of the memory. 
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Correlation follower for tracking an object 
Technical field 

The present invention relates to a correlation follower for track- 
ing an object, comprising an image sensor having a limited field 
of view and arranged to scan said field of view cyclically and to 
5 supply, in preselected form, a video signal representing the field 
of view, a video correlator with two addressable memories, one of 
which has the object of storing in digital form for each scanning 
cycle one section of the field of view, while the other memory is 
arranged to be updated with the content of the first memory, said 

10 video correlator being arranged to produce, once the image sensor 
has been so aligned that an object is encompassed within the sec- 
tion, an error signal controlling the alignment of the image sen- 
sor with the object, said error signal corresponding to a dis- 
placement of the section in the first memory relative to the sec- 

15 tion in the other memory, for which displacement a maximum corre- 
lation is achieved between the contents of the memories. 

Background art 

By correlation one can get, according to a predetermined rule of 
20 evaluation, a measure of how well the contents of the memories 
coincide at different relative displacements. 

Essential for the correlation and hence also for the tracking 
process is the way in which the other memory, subsequently referr- 

25 ed to as the reference image memory, is updated. At known corre- 
lation followers, see e.g. US Patent No. 3 828 122, updating is 
brought about in the course of one single scanning cycle, either 
periodically the updating process being repeated after a certain 
number of cycles, or when the maximum correlation drops below a 

30 preselected value. If an image element irrelevant to the track- 
ing process appears momentarily in the section the former updat- 
ing alternative entails the danger of the reference image memory 
being updated precisely when the image element appears, which 
may jeopardise the tracking process. With the other updating 
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alternative the appearance of the image element may trigger up- 
dating of the reference image memory. This leads in practice to 
tracking of the irrelevant image element. 

5 The object of the present invention is to bring about such updat- 
ing of the reference image memory that the above disadvantages 
are avoided and this is enabled in that the video correlator has 
means arranged -to select at each scanning cycle addresses of posi- 
tions in the. latter memory for the purpose of updating said memory 
10 randomly or according to a predetermined rule of selection so that 
said memory at each scanning cycle is updated only partly and in 
positions that are distributed over the area of the memory. 

Description of the drawing 

15 The invention is further explained below with reference to the 

attached drawing in which fig.l is a block diagram showing schema- 
tically the design of a correlation follower, fig. 2 is a block 
diagram showing the design of a video correlator and fig. 3 is a 
block diagram showing how updating is effected according to the 

20 invention. 

Description of a preferred embodiment 

In fig. 1 an image sensor of known type and consisting of a TV 
or IR camera with a field of view restricted in space is design- 
ated 1. The sensor is mounted on a platform 2 capable of being 
adjusted both sideways and in height and is so designed as to 
scan the field of view cyclically and to supply a video signal 
which reflects the image content within the field of view in 
electric form. The video signal is supplied, via a video corre- 
lator 3 connected with the sensor 1, to a monitor 4 on screen 5 
of which the sensor's field of view is displayed. A cursor is 
superimposed over the video signal in the video correlator 3, 
the position of which on the monitor screen 5 is shown as a 
window 6 which an operator can move, with the aid of a control 
lever 7, to any position on the screen. Also the size of window 
6 can be varied with the aid of control lever 7 and the said size 

f OMPI 



25 



30 



35 



wo 79/00260 



3 



PCT/SE78/00067 



so selected that the image 9 of an object 8 on screen 5 fits 
exactly within the window. In this way the effect of inter- 
ference from the surroundings of the object can be minimised, 
i.e. irrelevant background contrasts are screened off. Using 
5 control lever 7 the operator can also align the sensor 1 with 

object 8 and get the correlation follower to lock on the latter* 
In this process aligning signals from control lever 7 are trans- 
mitted via the video correlator 3 to an electronic platform 
system 10 in. which the aligning signals are converted into control 

10 signals for aligning the platform 2 and hence the sensor 1 both 
laterally and in height. After locking on, the sensor 1 tracks 
the movements of the object 8, whereby. an error signal is extract- 
ed from the video signal of sensor 1 by means of a correlation 
process, which will be discussed in connection with fig. 2, the 

13 said error signal being converted in the electronic platform 

system 10 into control signals for aligning the sensor 1 as de- 
scribed above in connection with control lever 7. 

In fig. 2 twin-line arrows illustrate a flow of image information. 
20 This flow of information reaches an analog/digital-converter 11 
from an image sensor of the above described type which is not 
shown in the figure. The A/D-converter 11 is designed to convert 
an analog video signal from the image sensor, the amplitude of 
which corresponds to the contrast at each point within the field 
25 of view of the sensor, to a digital signal containing in binary 
coded form the same data as the video signal. A part of the 
digital signal corresponding to the said section of the field of 
view of the sensor is read, during each scanning cycle, into a 
memory 12, designated henceforth as the real time memory. The 
30 reason for the designation "real time memory^' consists in the 

fact that the information stored in the real time memory in real 
time corresponds to the image content in the scanned section. 
From the real time memory 12 the flow of image information passes, 
on the other hand, via an updating date 13 to a memory 14, which 
35 is the above-mentioned reference image memory, and on the other 
hand to a correlation computing circuit 15 which also receives 



wo 79/00260 • PCT/SE78/06067 



image information from the reference image memory 14 -forming an 
output from the latter. In the same way the designation "refer- 
ence image memory" points to the fact that this memory is to 
serve as a reference during the correlation process. The flow 
5 of image information is controlled by a control logic 15 in 

accordance with the result of the correlation circuit measure- 
ment as will be described below. 

As previously stated the image sensor can be made to lock on to 
10 the object, by means of the control lever 7. When this is the 
case the content of the real time memory 12 is copied, during 
one scanning cycle, into the reference image memory 14. In the 
course of each scanning cycle, i.e. with each image read into 
the real time memory 12, the contents of the two memories are 
15 compared in the correlation computing circuit 15, the said con- 
tents being placed in different positions relative to one another. 
The comparison may be effected in accordance with any known 
method of correlation by means of which a factor of merit is 
calculated for each relative position of the image information 
20 in the two memories 12, 14. The relative position in which the 
highest factor of merit, i.e. the maximum correlation, occurs, 
is stored in a memory 17. Depending on the relative position in 
which maximum correlation is achieved the control logic 16 controls 
the flow of image information, i.e. the correlation computing 
25 process and the updating of the reference image memory 14. The 
control logic 16 has also the object to regulate, in response 
to signals from the control lever 7, the arrangement of window 6 
and to supply error signals to the electronic platform system 10. 



30 



Updating in accordance with the invention of the reference image 
memory 14 will now be explained with reference to fig. 3 in which 
as before twin-line arrows illustrate the flow of image informa- 



tion. 



35 In fig. 3 which shows especially how the real time memory 12 and the 
reference image memory 14 are addressed and the latter memory 
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is updated, 18 designates an address counter by means of which 
partial elements in the memories 12, 14 are addressed sequenti- 
ally via the address correction circuits 19 och 20, respectively, 
so that these are passed through line for line until all partial 

-5 elements have been covered. The address correction circuits 19, 
20 are controlled by memory 17, as regards the relative displace- 
ment between the contents of the memories at which the highest 
factor of merit is achieved as described above. The amplitude 
values in the partial elements addressed during this process in 

10 the respective memory are read into an amplitude logic 21 or a 

balancing circuit in which, according to a special characteristic 
of the invention, the amplitude values in corresponding positions 
are combined with one another, it being stated according to a 
criterion applying to each combination of amplitude values with 

15 which amplitude value the addressed position in the reference 

image memory 14 is to be updated. Such a criterion may be, for 
instance, that with quick or large changes in contrast, i»e* with 
large amplitude differences between the contents in the addressed 
partial elements, a mean value should be formed by means of which 

20 the partial element in the reference image memory is updated. 

This is equivalent to a certain filtration which prevents track- 
ing of any image element which suddenly appears in the field of 
view. If the amplitude values in two corresponding positions 
are equal, updating will of course take place with this value, 

25 i.e. the content of the partial element in the reference image 
memory remains unchanged. 

According to the primary characteristics of the invention updating 
shall occur randomly or according to a predetermined rule of 

30 selection so that during each scanning cycle the memory is up- 
dated only partly and in positions that are distributed over the 
area of the memory. This is achieved by that the amplitude value 
stated in accordance with the above criterion being fed to the 
addressed partial element in the reference image memory 14 via 

35 a gate 22 controlled by a prime or random number generator 23. 

With prime number generation the control operates in such a way 

( OMPI 
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as to ensure that the gate 22 is opened for each p-th of the 
partial elements addressed by the address counter 18, p being 
a prime number the size of which is selected with a view of the 
required updating rate. Should p be set to 1 this would mean 
that the entire reference image memory 14 would be updated during 
one single scanning cycle as described above. Therefore the 
prime number is at least equal to 3. With random number genera- 
tion the gate 21 is opened once the address counter 17 has counted 
forward s partial elements, s being a random number, e.g. from 
a table of random numbers. Whenever the gate 21 is opened a new 
random number is supplied. In this case the updating rate varies 
owing to selection of different mean values for the table of random 
numbers. 

By that the updating is carried out according to the invention 
and thus is neither related to the result of the correlation nor 
carried out periodically, it is achieved that with great probabi- 
lity the reference image memory Will not contains disturbing image 
elements and such that suddenly occur in the field of view of the 
sensor. This means that a correlation follower, the updating of 
the reference image memory of which is carried out as described 
above, is difficult to disturb and therefore the tracking of an 
object can be carried out with high accuracy. 

It is obvious that the invention can be modified in many ways 
within the scope of the inventive idea. It is possible, for 
instance, to utilize the video signals from a radar station for 
tracking an object. Further the updating may be carried out 
according to some other rule of selection than described above, 
e.g. according to a fixed pattern that is moved successively 
over the area of the memory. 
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is updated, 18 designates an address counter by means of which 
partial elements in the memories 12, 14 are addressed sequenti- 
ally via the address correction circuits 19 och 20, respectively, 
so that these are passed through line for line until all partial 

.5 elements have been covered. The address correction circuits 19, 
20 are controlled by memory 17, as regards the relative displace- 
ment between the contents of the memories at which the highest 
factor of merit is achieved as described above. The amplitude 
values in the partial elements addressed during this process in 

10 the respective memory are read into an amplitude logic 21 or a 

balancing circuit in which, according to a special characteristic 
of the invention, the amplitude values in corresponding positions 
are combined with one another, it being stated according to a 
criterion applying to each combination of amplitude values with 

15 which amplitude value the addressed position in the reference 

image memory 14 is to be updated. Such a criterion may be, for 
instance, that with quick or large changes in contrast, i,e. with 
large amplitude differences between the contents in the addressed 
partial elements, a mean value should be formed by means of which 

20 the partial element in the reference image memory is updated. 

This is equivalent to a certain filtration which prevents track- 
ing of any image element which suddenly appears in the field of 
view* If the amplitude values in two corresponding positions 
are equal, updating will of course take place with this value, 

25 i.e. the content of the partial element in the reference image 
memory remains unchanged. 

According to the primary characteristics of the invention updating 
shall occur randomly or according to a predetermined rule of 

30 selection so that during each scanning cycle the memory is up- 
dated only partly and in positions that are distributed over the 
area of the memory. This is achieved by that the amplitude value 
stated in accordance with the above criterion being fed to the 
addressed partial element in the reference image memory 14 via 

35 a gate 22 controlled by a prime or random number generator 23. 

With prime number generation the control operates in such a way 
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as to ensure that the gate 22 is opened for each p-th of the 
partial elements addressed by the address counter 18, p being 
a prime number the size of which is selected with a view of the 
required updating rate. Should p be set to 1 this would mean 
5 ' that the entire reference image memory 14 would be updated during 
one single scanning cycle as described above. Therefore the 
prime number is at least equal to 3. With random number genera- 
tion the gate 21 is opened once the address counter 17 has counted 
forward s partial elements, s being a random number, e.g. from 
10 a table of random numbers. Whenever the gate 21 is opened a new 
random number is supplied. In this case the updating rate varies 
owing to selection of different mean values for the table of random 
numbers. 

15 By that the updating is carried out according to the invention 

and thus is neither related to the result of the correlation nor 
carried out periodically, it is achieved that with great probabi- 
lity the reference image memory Will not contains disturbing image 
elements and such that suddenly occur in the field of view of the 

20 sensor. This means that a correlation follower, the updating of 
the reference image memory of which is carried out as described 
above, is difficult to disturb and therefore the tracking of an 
object can be carried out with high accuracy. 

25 It is obvious that the invention can be modified in many ways 
within the scope of the inventive idea. It is possible, for 
instance, to utilize the video signals from a radar station for 
tracking an object. Further the updating may be carried out 
according to some other rule of selection than described above, 

30 e.g. according to a fixed pattern that is moved successively 
over the area of the memory. 
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1. A correlation follower for tracking an object, comprising 
an image sensor having a limited field of view and arranged 
to cyclically scan said field of view and to supply, in a 
preselected form, a video signal corresponding to the field 
of view, and a video correlator with two addressable memories 
one of which has the object of storing in digital form for 
each scanning cycle a section of the field of view while the 
other memory is arranged to be updated with the contents of 
the first memory, said video correlator being arranged to 
produce, once the image sensor has been so aligned that an 
object- is encompassed within the section, an error signal 
controlling the alignment of the image sensor with the object, 
said error signal corresponding to a displacement of the sec- 
tion in the first memory relative to the section in the other 
memory for which displacement a maximum correlation is achiev- 
ed between the contents of the memories, characte r-.. 
i s e d ..in that the video correlator (3) has means (21, 22) 
arranged to select, for the purpose of updating the other 
memory (14), addresses of positions in the latter memory 
randomly or according to a predetermined rule of selection 
so that the other memory (14) for each scanning cycle is up- 
dated only partly and in positions that are distributed over 
the area of the memory. 

2. A correlation follower according to claim 1, character- 
ised in that, the video correlator (3) is arranged to 
select the addresses to the positions of the memory for the 
updating so that of addresses generated in a certain sequence 
each p-th is selected where p is a prime number. 

3. A correlation follower according to claim 1, character 
i s e d in that the video correlator (3) is arranged to read 
while updating each selected position in the other memory (14) 
the contents stored in that memory and at the corresponding . 
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position of the first memory (12), to combine the contents of 
the said positions with one another and to state, in accordance 
with the criterion applying to each combination of contents, 
with which value the selected position in the other memory (14) 
5 is to be updated. 
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